FIG. 1 is a schematic of a related art user device 100 illustrating components thereof that may permit creation of immersive content, such as augmented reality (AR) 101. AR applications may use an overlay of graphical/added subject matter on live or recorded video or still images presented on device 100. For example, a user or application may position a static graphic, text, or other visual element superimposed on the underlying video or image. Augmented reality applications may also augment underlying visual data by changing or replacing select elements in the underlying environment and tracking those elements as if they are actually present in the underlying environment, or use the underlying visual data to position the augmented reality subject matter as displayed. For example, an underlying object may be recognized in a video feed or image still of an environment, and the augmented reality application may superimpose additional elements on or apply coloration, designs, distortions, etc. to, or relative to, the object, so as to further the augmented reality effect that the object and surrounding environment actually includes such elements or characteristics.
To handle AR, device 100 may include a camera package 110 including a lens, image sensor, and/or microphone, a computer processor 120, persistent and/or transient storage 130, external communications 140, display 180, AR configuration 170, and/or input device and input sensors 185. Although elements are shown within a single device 100, it is understood that any element may be separate and connected through appropriate communications such as an external bus for a peripheral or wired or wireless connection. Processor 120 may include one or more computer processors connected to and programmed or otherwise configured to control the various elements device 100. Processor 120 may further be configured to create, transmit, and/or perform augmented reality 101 in accordance with programming or input and may potentially include an associated processor cache, transient memory, video buffer, etc., configured or programmed to processes AR 101. For example, AR configuration 170 may include software and/or firmware that instruct processor 120 how to create or display AR 101 received from an application or outside source. Processor 120 may also receive sensor information from sensors 185, e.g., touch or cursor information, and process the same as user interaction or input. Processor 120 may further execute software or include configured hardware that allows for execution of example methods discussed below.
Storage 130 may be a dedicated data storage drive or may be a partition of a general data store in which augmented reality information, origin or limitation information, application information, and/or device operations and raw data can be saved. Storage 130 may be, for example, random access memory (RAM), read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), flash memory, a hard disk, a processor cache, optical media, and/or other computer readable media.
Camera 110 may include one or more lenses and/or apertures that may be controlled by actuators that move the lenses and apertures among different positions to focus captured optical data. Similarly, camera 110 may adjust focus digitally or in response to user input defining focus locations in the scene being captured. Camera 110 may include image sensor elements such as a charge coupled device (CCD) array, a photodiode array, or any other image sensing device that receives light, potentially via the lens, and generates image data in response to the received light. Camera 110 may include a light to aid in reflection and/or focusing laser. Camera 110 may be further configured to obtain or adjust image information such as focus, zoom, white balance, exposure, saturation, and/or other image functions. Camera 110 and/or processor 120 may be further configured with one or more video codecs or other image processing software or drivers to capture, process, and store external independent media such as actual video from the environment as well as augmented reality. An included microphone may be any auditory transmission and/or reception device capable of audio pickup and/or playback.
Display 180 may be a screen, viewfinder, monitor, projector, glasses front, or any other device capable of visually displaying visual augmented reality 101. For example, display 180 may be a touchscreen on a smartphone like iOS or Android devices or on a tablet like an iPad or Surface, or display may be an LCD monitor or projector, for example. Sensors 185 provide input information, potentially as a part of display 180 and/or as a separate component. For example, if display 180 is a touchscreen, sensors may be embedded multi- or single-touch capacitive sensors capable of detecting finger or stylus touch, pressure, movement, etc., with respect to display 180. Or for example, sensors 185 may be an accelerometer or magnetized compass with associated hardware or software capable of determining device orientation and/or movement. Or for example, sensors 185 may be a button or an external mouse or joystick and associated hardware or software capable of controlling and determining cursor position and/or activation with respect to display 180 during operation of device 100. Sensors 185 are connected to processor 120 and can deliver sensed input information to processor 120 with respect to display 180, including cursor or contact position, duration, numerosity, pressure, movement speed, etc.
Device 100 may further include a communications port 140 for external wired or wireless communication. For example, communications port 140 may be an antenna configured to transmit and receive on CDMA bands, a Wi-Fi antenna, a near field communications transmitter/receiver, a GPS receiver, an external serial port or external disk drive, etc. Processor 120 may provide data from storage 130, input data from camera 110, sensors 185, etc., to external devices through communications port 140, as well as receive application and/or augmented reality and other information from providers through port 140. Further, communications port 140 may function as another input source for sensors 185.
Device 100 may be mobile, such as a laptop, smartphone, wearable, dashcam, camcorder, GPS device, etc., or device 100 may be relatively immobile, such as a desktop computer, wide-area network, fixed security camera, etc. Although networked elements and functionalities of device 100 are shown in FIG. 1 as individual components with specific groupings and subcomponents, it is understood that these elements may be co-located in a single device having adequately differentiated data storage and/or file systems and processing configurations. Alternatively, the elements shown in FIG. 1 may be remote and plural, with functionality shared across several pieces of hardware, each communicatively connected at adequate speeds to provide necessary data transfer and analysis, if, for example, more resources or better logistics are available in distinct locations.